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ABSTRACT 



Dudley knox library 
WVA l POSTGRADUATE SCHOOL 
MONTEREY, CA 93940 



In partial response to a memorandum for the Superin- 
tendent, Naval Postgraduate School, from the Director of 
the Fiscal Management Division, Office of the Chief of 
Naval Operations (Serial 922E21/587 52 6 dated 24 April 1978), 
this thesis attempts to analyze the historical relationship 
between the Navy's investment in current RDT&E and future 
investment in procurement. Utilizing data from fiscal 
years 1962 through 1979 and single equation econometric 
forecasting techniques, linear models predicting procurement 
one to four years in advance based on current RDT&E are 
developed. From time-series data, with the models adjusted 
for serial correlation of the error terms, ex post forecasts 
and confidence interval estimations are used to evaluate 
the extent and usefulness of the predictive relationships 
discovered . 

Eight separate models are developed, and analysis of 
results indicates the existence of a predictive relation- 
ship. However, there are also indications that the basic 
relationship may have changed during the period under study. 
The relative inaccuracy of forecasting methods when earlier 
data are ignored makes the usefulness of these procedures 
to those who shape future Navy budgets difficult to 
determine . 
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I. 



INTRODUCTION 



The analysis to be conducted in this thesis was 
originally proposed by a memorandum from the Director 
of the Fiscal Management Division, Office of the Chief 
of Naval Operations, to the Superintendent of the Naval 
Postgraduate School. ^ Appendix A contains a copy of that 
correspondence, which expressed concern that more and more 
promising new weapons programs were being terminated 
because their estimated procurement costs made them 
unaffordable. At the same time, the Department of 
Defense was continuing to increase its research and 
development efforts. For example, the fiscal year (FY) 

1979 Navy Research, Development, Test, and Evaluation 
(RDT&E,N) appropriation request provided a real program 
growth of about six percent over the FY 1978 level. The 
memorandum implied that the increasing complexity and costs 
of modern systems , at least in part, resulted from earlier 
research and development effort; and the memorandum expressed 
an interest in developing, if possible, a predictive model. 



Although the correspondence is officially from the 
Office of the Chief of Naval Operations (OPNAV) only, its 
author and most of his subordinates also hold positions in 
the Office of the Navy Comptroller (NAVCOMPT) under the 
Secretary of the Navy. To avoid confusion, future refer- 
ences to this correspondence, the analysis or the persons 
who conducted it will use the modifier "OPNAV/NAVCOMPT" . 
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Specifically, based upon the current investment in research 
and development, can future procurement requirements be 
predicted? The OPNAV/NAVCOMPT staff conducted a preliminary 
study in pursuit of this question, and a discussion of their 
analysis follows. 

A. THE OPNAV/NAVCOMPT ANALYSIS 

The analysis conducted by the OPNAV/NAVCOMPT staff 
was limited to eighteen individual development/procurement 
programs included in the FY 1979 budget submission. These 
programs were all in the engineering development stage and 
had progressed to the point of having budget quality esti- 
mates of both total research and development and procurement 
costs through the program years (the four years beyond the 
budget year) [Refs. 4 and 5]. For each of these programs 
a ratio of total procurement costs to total research and 
development costs was calcuated. This ratio was then 

adjusted or weighted by the program's fraction of the 

2 ... 

FY 1979 RDTS<E,N sample. The sum of these individual 
weighted factors then equated to the relationship between 
research and development and procurement costs : for every 

dollar of RDT&E,N, the Navy could expect to spend approxi- 
mately four dollars procuring the associated weapons systems. 



2 

Weighted Factor = Procurement to RDT&E,N Ratio x 
[Program FY 79 RDT5E , N -j- Total Sample RDT&E , N ] . 
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The limitations of such a study are more or less 
obvious. The study addresses only a single year's data, 
and much of those are estimated. There exists a wide 
variance among the calculated procurement to research and 
development ratios (from a high of 33.2 to a low of 0.4). 
Thus, what may be true for this sample may not necessarily 
be true for a different one. The amount of RDT&E,N money 
included in the study represented only 25 percent of the 
total RDT&E,N budget request. Finally, the ratios of 
procurement to research and development were apparently 
calculated without adjusting costs to consistent or con- 
stant dollar figures. The study mixed dollars of past, 
present, and projected future purchasing power. Adjusting 
for this difference would not be an easy task in light of 
the way in which the data were presented. 
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II. AN EXPANDED STUDY 



This thesis will expand the study of the relationship 
between procurement and research and development costs. 
Investigation of possible methods of conducting this 
expanded analysis produced two general alternatives: 

A. ALTERNATIVE METHODS 

1 . The Micro Approach 

This alternative would proceed in an effort to 
identify individual programs for which RDT&E , N and procure 
ment investments had been made. These data could then be 
aggregated to provide a basis for estimating the overall 
relationship of procurement to research and development in 
much the same manner as the OPNAV/NAVCOMPT analysis. In 
addition, individual programs could then be categorized 
in such a manner as to study particular types of systems 
(e.g. high performance aircraft or surface radar systems). 

2 . The Macro Approach 

This alternative would use aggregate historical 
budget data rather than figures for individual programs. 
The relationship of total RDT&E, N to a total of the Navy's 
procurement accounts could be studied, as well as relation 
ships by program budget activity or Department of Defense 
(DOD) program category (e.g. RDT&E, N for aircraft and 
missiles versus procurement of aircraft and missiles). 
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The macro approach was chosen for several reasons. 
First, pursuit of the micro approach with the thought of 
logically connecting each dollar spent in RDT&E , N to an 
item of current or future procurement would have fallen 
well short of that goal. There is a significant portion of 
the total RDT&E, N budget, such as basic research and manage- 
ment and support, which could never be "applied" to a 
particular procurement action in other than an arbitrary 
manner. Secondly, what historical data we now have con- 
cerning the RDT&E, N and procurement monies spent on 
specific programs are made available mainly through Selected 
Acquisition Reports (SAR's) which are submitted by the Navy 
to the Department of Defense. These SAR's provide a means 
of tracking costs on a regular basis. Unfortunately, these 
reports cover only major programs (over $75 million RDT&E, N 
and/or $300 million procurement) and go back only as far 
as 1970 [Refs. 3 and 6]. Finally, an attempt to trace 
historical RDT&E, N detailed program element data to pro- 
curement data for the major programs or for the myriad of 
smaller programs would have resulted in an excessively 
time-consuming data extraction process and, in the end, 
would still have left much of the RDT&E, N investment 
unaccounted for. 

B. ASSUMPTIONS AND DEFINITIONS 

The most basic assumption underlying any financial 
analysis is that human behavior can be reasonably and 
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accurately represented by a dollar figure. More specif- 
ically in regard to this study, human behavior which we 
might define as the "level of effort" in research and 
development and the "level of effort" in procurement are 
assumed to be susceptible to representation by dollar 
figures. But which dollar figure among the many available 
in the DOD budgetary process best represents these levels 
of effort? Terms like authorization, appropriation, outlay, 
expenditure, budget authority, new obligational authority, 
total obligational authority, budget activity, program, 
total program, DOD program category, reimbursements, reim- 
bursable, budget plan, and obligation are not easily, 
widely, or fully understood, in spite of the many text 
book definitions available. 

Appendix B is a glossary which will provide the reader 
with definitions of terms as they are used in this analysis, 
but additional explanation of this thesis ' answer to the 
question posed above (i.e. which dollar figure is best) is 
in order. Table 1 presents a page from The Budget of the 
United States Government, 1970 - Appendix and will serve 
as an example of the complexity (and in some cases inconsis- 
tency) of terminology and as support for the answer at which 
we will arrive. 

Table 1 presents the schedule of program costs and 
financing for the Navy's procurement of aircraft and mis- 
siles (PAMN) appropriation account for FY 1970. This 
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TABLE I 



Pxocu&cmxnt of AiacRArr ajto Miesiixs, Navt — C ontinued 

Prof r»m and Financing (in thousands of doflart) 



Budget plao Obligation* 

IdeatiAcatioa coda (srnouota tor procurecneot actions programed) 

07-15-1 $05-0- 1—05 1 * , — - — - "• • — ■ . — 

1968 actual 1969 nuatu 1070 aatimat* 1968 actual 1969 eniroaui 1970 eatimat* 



Program by activities: 

Direct; 

1. Combat aircraft ....... 1.535,275 1,503.340 

2. Airlift aircraft 15,039 

3. Trainer aircraft. 73.798 33.415 

4 Other aircraft 4.263 ............ 

5. Modification of a u-ct aft... ........... 441,062 401.251 

6. Aircraft spares and repair parti 606,066 373,626 

7. Aircraft support equipment and facilities ................ 99.301 101,500 

3. Ballistic mmilei 175.039 384.660 

9. Other missiles 329.203 326.941 

!(X Modification of missiles ... .... 20.820 19.345 

fl. Missile spares and repair parts 25.743 21,971 

12. Missile support equipment and facilities __ 29,966 16,779 



Total direct 3.340.538 3.248.387 



Reimbursable: 

7. Aircraft support equipment and Facilities.. ...... ........... 29,281 26,000 

12. Missile support equipment and facilities.. 17.740 21,000 



Total reimbursable 47.021 47.000 



10 Total 3.387,559 3.295.387 

Financing: 

Receipts and reimbursements from: 

11 Federal funds -35.310 - 35.300 

13 Trust funds -22.317 -22,300 

14 Non-Fedcral sources ; ... — 4,394 —4,400 

21 Unobligated balance available, start of year: 

For completion of prior year budget plans ... .... .... 

Available to finance new budget plans ... —159.000 

Reprograming from prior y-sr budget plans ................... —243,638 — 3Q.037 

22 Unobligated balance transferred from other secounts.... ............ —410,000 

24 Unobligated balance available, end of year: 

For completion of prior year budget planj__. ...... ............ 

Available to finance subsequent year budget plans 159.000 



1,494,100 


1.683.371 


36,800 




I 10.200 


81.322 


8,600 


4,144 


325.900 


469.680 


584.200 


632,909 


99.100 


96.877 


517.900 


99.696 


253.000 


335,239 


27.000 


21.275 


38.900 


13,275 


23.300 


20.023 


3,524.000 


3.457,311 


26.000 


14.535 


21.000 


19,357 


47.000 


34,442 


3.571.000 


3.492.253 



-35.300 


-19,233 


-22,300 


— 19.539 


-4.400 


-3.991 



-1.572.837 



— 75.000 

-25.000 - 7.517 

1,212.814 

159.000 



BudfH authority 3,240,900 2,574,300 3.409.000 3,240,900 



Budget authority: 

40 Appropriation 2,939.100 2.574.300 3.409.CC0 2.939.100 

42 Transferrer from other accounts .... 301,300 ....... 301,300 



1.484.600 


1.570.300 


9,300 


27,700 


83,200 


104.300 


II.OCO 


10,600 


105.200 


361,^00 


390,000 


553.400 


88.100 


97.300 


330.500 


485,000 


325.400 


232.400 


15.400 


19,900 


20.300 


27. COO 


16,600 


22.600 


3.180.600 


3,512.400 


15.000 


15.000 


20.000 


20,000 


35.000 


35.CCO 



3,215,600 * 3.547.400 



-35.300 

-22.300 

-4.400 


—35.500 

-22.300 

—*.400 


-1.212.814 

-159.000 


—1,202,514 


-410.000 


-25.000 


1,202,514 


1.151. K4 





2,574,300 


3. -109 ,000 


2,574.300 


3,409,000 



43 Appropriation (adjusted). 



3,240,900 2,S74,3uO 3,409,000 3,240,900 2,574,300 3,409,000 



Relation of obligations to outlays: 

71 Obligations incurred, net ... - 3,449.440 3.153,600 3,485,400 

72 Obligated balance, start of year.. ... ........ — ... — — — - 3,764.367 3.571,799 3 , 1 .0, j99 

74 Obligated. balance, end of year ... ... ...... ........ —3,571,799 — 3,120,399 —3,265,799 



90 Outlays - 3.642,008 3,605,000 3,340,000 



‘ Reimburse mcoti from aoo-Federal *ourec» are deneed prineipally from «•*!» die* to <o»ei»o (ore mutants 
•i ofticr aircraft component*. and tparea and repair pare* (72 L'i.C, 23)3). 
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schedule provides actual figures for the previous year 
(1968) and estimates for the current year (1969) and budget 
year (1970). It is presented in three main parts: (1) the 

program-by-activities section, (2) the financing section, 
and (3) the section on relation of obligations to outlays. 

The program-by-activities section presents the results 
of the executive branch's programming function in terms of 
financial requirements by budget activity. This classifi- 
cation by budget activity is not consistent throughout the 
budget for each appropriation nor is it consistent with the 
DOD classification by program category [Refs. 2, 5, and 7] . 
The former characteristic reflects the fact that each 
appropriation account is divided among several budget 
activities and that these vary from appropriation to appro- 
priation. The latter indicates that DOD program category 
classification (e.g. strategic forces, general purpose 
forces, etc.) has an orientation entirely different from 
the appropriation's program-by-activities section. This 
necessitates the function known as "crosswalking", or being 
able to shift from budget appropriation to DOD program and 
back again [Refs. 1 and 4] . 

The financing section shows the sources of funds (budget 
authority, receipts and reimbursements, unobligated balances 
available and amounts transferred in, etc.) which will be 
used to support the financial requirements as determined 
above . 
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The relation-of-obligation-to-outlays section shows 
obligations net of offsetting collections at the start and 
end of each year and presents the actual net cash outflow 
as a result of the programming and budgeting activity 
conducted in connection with this appropriation account. 

The appropriation (adjusted) section is included when 
required for the year prior. It merely shows what portion 
of the appropriation for that year was actually transferred 
from other accounts . 

Selection of that dollar figure which would best repre- 
sent the level of effort is based upon the definitions of 
the terms programming and budgeting as they are used in the 
context of the Federal financial management process. Pro- 
gramming is defined as the process of translating planned 
military force requirements into time-phased manpower and 
material resource requirements. Budgeting is defined as 
the process of translating approved resource requirements 
(manpower and material) into time-phased financial require- 
ments [Ref. 5]. Where these processes interface, that is, 
where time-phased manpower and material resource requirements 
are first stated in terms of financial requirements appears 
to be the point at which to determine the best financial 
representative of "level of effort". The dollar figure at 
this point (refer to Table 1) is the total direct program 
figure and represents the financial requirements of each 
budget activity necessary to support approved programs. 
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This total is the equivalent of the more commonly-used 
term, total obligational authority (TOA) . However, it is 
not simply new obligational authority plus unobligated 
balances, which is the most frequently heard definition of 
TOA [Ref. 1] . 

For the purposes of this study, the data used will be 
the total direct program by budget activities for the 
Navy's Research Development, Test, and Evaluation (RDT&E,N) , 
and the Navy's procurement accounts (PAMN - Procurement of 
Aircraft and Missiles; WPN - Weapons Procurement; APN - 
Aircraft Procurement; OPN - Other Procurement; and PMC - 
Procurement, Marine Corps) . These data will be actual 
historical data in so far as practicable. For fiscal 
years 1978 and 1979 the data will be estimates only [Ref. 8]. 

C. HYPOTHESIS 

The general working hypothesis under which the remainder 
of this analysis will be conducted is as follows: there 

exists a predictive relationship between current Navy 
research and development efforts and future investments 
in procurement. 
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III. THE METHODOLOGY 



A. THE DATA 

As indicated above, data used in this study will be 
the total direct program figures for the Navy's RDT&E and 
procurement accounts. These have been taken from the 
annually published Budget of the United States Government 
. . .Appendix [Ref. 8] and, except for FY 1978 and FY 1979, 
will be actual figures. That is, the amounts will originate 
in the budget plan's "actual" column of the program-and- 
financing section for each appropriation. For example, the 
figure for the Navy's 1968 PAMN account is obtained from 
The Budget of the United States Government, 1970 - Appendix . 
The total direct FY 1968 PAMN program is $3,340,538,000, 
as shown in Table 1. 

Data were collected for each full fiscal year starting 
with 1962 and ending with 1979. It was felt that this 
time span would provide enough data to develop a predictive 
model if, in fact, such a relationship existed. Of course, 
the figures contained in the budget documents are in current 
U.S. dollars, and an adjustment to constant dollars (FY 
1979 dollars in this case) was made prior to the conduct 
of the analysis. Factors for converting current dollars to 



^Data from the transition quarter of 1976 , designated 
19TQ, were ignored by this study. 
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constant FY 1979 dollars were obtained for each appropria- 
tion account as promulgated by DOD on February 10, 1978. 
Appendix C contains in tabular form the current dollars, 
adjusting factors, and constant dollars for all of the 
quantitative data used in this study. 

B. THE MODEL 

The basic principles of econometrics and economic fore- 
casting will be used to determine if there exists a pre- 
dictive relationship between current efforts in RDTS.E and 
future efforts in procurement. This relationship will be 
described initially by the classical normal linear regression 
model : 

Y fc = a + b X t + e t (1) 

where 

Y = the dependent variable, 

X = the explanatory variable, 

e = a random error term whose values are based 

upon an underlying probability distribution, 

a and b = regression parameters which are unknown, 

X and Y are observable but e is not, and 

th 

the subscript t refers to the t 1 observation 
[Ref. 10] . 
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In this analysis the explanatory variable, X, will 
always be the RDT&E,N direct program in the budget year 
(BY) , while the dependent variable will be one or a total 
of the Navy's procurement direct programs for one or more 
of the program years. Since these observations are made 
over time, they are often referred to as "time-series data" 
[Ref. 10]. Thus, the subscript t is used rather than the 
more common i . 

C. ASSUMPTIONS 

In addition to the general form of the model as expressed 
in equation (1), the classical normal linear regression 
model must conform to the following basic assumptions. 

1 . Normality 

The error term, e , is normally distributed. 

2 . Zero Mean 

The expected value of the error term, E(e t ), is 
equal to zero. 

3 . Homoscedasticity 

Every disturbance, e fc , has the same variance, S , 
whose value is unknown. 

4 . Nonstochastic X 

The explantory variable, X , is nonstochastic 
(nonrandom) with values fixed in repeated samples and such 
that, for any sample size, 

n 

± l (X t -X) 2 
t=l 
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is a finite number different from zero. (X is the mean 
value of all X , and n is the number of data pairs in the 
sample. ) 

5 . Nonautoregression 

The error terms, e , are uncorrelated in a statis- 
tical sense [Refs. 10 and 12]. 

This analysis assumes that the data conform to all 
of the basic assumptions listed above except the fifth, 
nonautoregression. This assumption is most often violated 
by relations estimated from time-series data [Refs. 9 and 
10], and the analysis will proceed on the opposite assump- 
tion, determine the extent of autoregression or serial 
correlation present, and make the necessary adjustments to 
the model. 

D. PROCEDUREAL DETAILS 

For each set of data, certain steps will be taken in 
the following logical sequence. 

1 . Ordinary Least Squares (OLS) Regression 

The method of least squares will be used to provide 
estimates of the model's regression parameters a and b 
where 

h t wx 



and 
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b 



^ x t Y t - ? x t l\ 
nZ * t 2 - dx t ) 2 

This procedure in graphic terms results in a straight line 

which minimizes the sum of the squared deviations of the 

data points on the graph from the points on the straight 

line (with distances measured vertically) . 

2 . The Hildreth-Lu Procedure 

Autoregression of the data's error terms has been 

assumed. The next step is to determine the extent of first 

order serial correlation present, if any, by using the 

Hildreth-Lu procedure [Ref. 12] to estimate a new parameter, 

r, which is the correlation coefficient between errors in 

time period t+1 and errors in time period t. 

In this procedure, a set of values are specified 

for r, and for each value of r, another OLS regression is 

4 

conducted and error sum of the squares (ESS) calculated 
on the following equation. 

Y t * = a(l-r) + bX fc * 



where 



V 



Y t+1 rY t' 



4 ESS = The sum of the squared differences between the 
estimated and observed values of the dependent variable. 
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and 



V = 



x t+l - rx f 



The procedure selects that value of r which results in 
the lowest ESS, guarantees a maximum likelihood estimate 
of r, and may be repeated until the desired accuracy is 
achieved, to the nearest thousandth in this instance. 

3 . Generalized Differencing 

By this procedure the original model is changed into 
one for which the error terms are independent. The result 
is the transformed equation. 



Y * = a(l-r) + bX t * + v fc . 



( 2 ) 



where 



V 



= Y 



t+1 



- rY 



t' 



V 



= X 



t+1 



- rX 



t' 



and 



v. 



= e 



t+1 



- re. 



are the generalized differences of Y , X fc , and e fc , respec- 
tively. The transformed equation (2) has an error process 
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which is independently distributed with zero mean and 
constant variance, and an OLS regression applied to it will 
produce estimates of all the parameters. The intercept 
of the original model, however, must be calculated from 
che estimated intercept associated with equation (1) 

[Ref. 12]. Thus, once the correlation coefficient of 
the error terms, r, is known or has been estimated, it is 
a relatively simple matter to adjust the original model. 

4 . Forecasting 

Having completed the procedure as outlined above, 
the estimates of a(l-r) , b, and r as they appear in model 
(2) have been obtained. These are then substituted into 
the model along with expressions for the generalized 
differences of Y fc , X t , and e fc : 

Y t+1 - rY t = a(l-r) + b(X t+1 -rX t > + e fc+1 - « t 

A forecast value of Y denoted by Y fc+ ^ may then be 

obtained : 



Y 



t+1 



rY t + a(l-r) -r bX fc+1 - brX t 



( 3 ) 



The error term, v fc = e fc+1 - re fc , has been dropped from 
equation (3) for reasons of clarity. Without proceeding 
with mathematical proofs, the variance of the forecast 
error which will be used to obtain confidence intervals 

s\ 

about the point estimate (Y^ + ^) is given by 
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where 




( 1-r 2 ) 




[1 




(X 



t + 1 



- X) 




2 



S 2 = tT2 I« t -V ) 2 



( 4 ) 



(5) 



and 



T = the number of observed pairs in the 
original data set. 



THe 95% confidence interval using the Student's t distribu- 

A 

tion for Y t+ ^ is 



Y t+1 t .025;n-2 S v Y t+1 Y t+1 + t .025;n-2 S v 



( 6 ) 



where 



S = the standard error of the forecast (7) 



v 



2 2 2 
S Z = (1-r ) S/ 

v f 



and 
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^.025-n-2 = t ^ ie va l ue °f the t distribution with 

n-2 degrees of freedom such that 
2.5% of the area under the curve 
lies to the right of that value. 
Provides the basis for a two-sided 
hypothesis test at the 95% level 
of significance [Ref. 11]. 

From equations (3) and (4), the statistical accuracy of 
the prediction is dependent upon the size of the sample, 
the range of experience of the explanatory variable X, 
the distance between the new explanatory variable X J . +1 , 
and the average value of all X, X [Ref. 12]. 

5 . Presentations 

For each set of variables the following presenta- 
tions will be made : 

a. A table of observed variables, variables in 
generalized difference form, and ex post forecasts. Data 
will be rounded to the nearest million and expressed in 
FY 1979 dollars. 

b. An expression for the OLS regression in the 
form, Y = a + bX 

c. An expression of the new, corrected or adjusted 
regression in the form Y fc+ ^ = rY^ + a(l-r) + bX t+ ^ ~ brX fc 

d. A table of data which will include for a range 

of values of the explanatory variable (X / the point 

estimate cf the dependent variable (Y from equation (3) , 

o 

the variance of the forecast error (S .") from equation (4) , 

r 

the standard error of the forecast (S ) from equation (7) , 

v 

and the width of one side cf a 95% confidence interval from 
equation ( 6 ) . 
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e. A graph with procurement (Y ) measured along 
the ordinate and RDT&E , N (X fc ) along the abscissa, displaying 
the data points, the OLS regression line, the adjusted 
regression line, and 95% confidence bands about the adjusted 
regression line. 

f. A graph of procurement and RDT&E measured along 
the ordinate versus time measured along the abscissa, dis- 
playing observed data, forecasted results (i.e. what the 
model would have predicted) , and 95% confidence bands about 
the forecasted value. The width of these confidence bands 
is that of the "next" forecast if the explanatory variable 
(X fc+ ^) were exactly equal to the average of the previous 
explanatory variables (X). Thus, it is as narrow as it 
can be and, for the purposes of presenting pictorially how 
well the model predicted ex post facto, presents the most 

rigorous test. In addition, a numerical expression of the 

2 

goodness of fit over the entire time period, R , is 
provided . 

2 _ Regression Sum of the Squares (RSS) 

Total Sum of the Squares (TSS) 

where 

TSS = The total variation of the dependent 

variable. The sum of the squares of the 
difference between the observed values 
of Y and the average value of Y fc , 



and 



27 



RSS 



= The explained variation of the dependent 
variable. The sum of the squares of 
the differences between the forecasted 
or predicted values of Y and the 
average value of Y^_. 

2 

R expresses that fraction of the total variance 
of the dependent variable which is "explained" 
by the explanatory variable. 
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IV. THE RESULTS 



The results of the analyses will now be presented as 
described in paragraph II. D. 5. The first four analyses 
will match budget year RDT&E with a single year of total 
Navy procurement one, two, three, and four years in the 
future. The fifth analysis matches budget year RDT&E 
with the aggregate of total Navy procurement in the program 
years, and the remaining three analyses explore alternatives 
along similar four-year aggregate procurement totals, again 
using RDT&E as the explanatory variable. 

For each set of variables, denoted by second-order 
headings (A,B,...,H), the following presentations will 
made under corresponding third-order headings: (1) A 

table of observed variables, variables in generalized 
difference form and ex post forecasts (more fully explained 
in paragraph III. D. 5. a) . (2) An expression for the OLS 

regression in the form 

Y = a + bX 

(cf., paragraph III.D.S.b). (3) An expression for the 
new regression in the form 



Y 



t+1 



rY + a(l-r) + bx t+l " brX t 



(cf . , paragraph III.D.S.c). (4) A table of data which 
includes for a range of values of the explanatory variable 
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(X t+1 ) , the point estimate of the dependent variable (Y t+ ^) / 
the variance of the forecast error (S f ) , the standard 
error of the forecast (S v ) , and the width of one side of 
a 95% confidence interval (cf., paragraph III.D.5.d). 

In addition, two figures, numbered sequentially, will be 
provided for each set of variables, the first showing 
pictorially the relationship between Procurement and RDT&E,N, 
and the second showing the relationship of both over time 
as well as the model's forecasted results (cf., paragraphs 
III . D . 5 . e and II. D. 5. f) . 



30 



A. PROCUREMENT IN BUDGET- YEAR-PLUS-ONE VERSUS RDT&E , N 
IN BUDGET YEAR 



1. Observed Variables, Variables in Generalized 





Difference Form 


(r 


= 0.462) 


, and Ex ] 


Post Forecast 


FY 


X 


X* 


FY 


Y 


Y* 


Y t + 1 


62 


3025 




63 


19628 




— 


63 


3394 


1996.45 


64 


16449 


7380.364 


16998 


64 


3473 


1904.972 


65 


14580 


6980 . 562 


15794 


65 


3087 


1482.474 


66 


18687 


11951.04 


16152 


66 


3332 


1905.806 


67 


19380 


10746.606 


16826 


67 


3961 


2421.616 


68 


15271 


6317.44 


15655 


68 


3684 


1854.018 


69 


13822 


6766.798 


15397 


69 


4075 


2372.992 


70 


14801 


8415.236 


13228 


70 


4011 


2128.35 


71 


13405 


6566.938 


14387 


71 


3718 


1864.918 


72 


14510 


8316.89 


14503 


72 


3865 


2147.284 


73 


13744 


7040 . 38 


14198 


73 


3841 


2055.37 


74 


12307 


5957 . 272 


14110 


74 


3755 


1980.458 


75 


10879 


5193.166 


13662 


75 


3893 


2158.19 


76 


12332 


7305 . 902 


13579 


76 


3991 


2192.434 


77 


14787 


9089 . 616 


13061 


77 


4297 


2453.158 


78 


15122 


8290 . 406 


13442 


78 


4266 


2280.786 


79 


13919 


6932.636 


14090 




2. 


OLS Regression: 


Y 


= 31195 


.8483 - 4. 


3461 X 




3 . 


Regression Line 


Adjusted for Serial 


Correlation : 




Y . = 


rY t + a(l-r) + 


bX 


. +1 - brX. 






t+1 






t 








Y t + 1 = 


( . 462) (13919) + 13700 .9173 


- 2.8905X t+1 + (2.8905) ( 




II 

i — 1 
+ 

< >* 


25828.35863 - 


2.8905 X n 








4. 


Data Table: 










X t + 1 


Y t + 1 


S f 


2 


S v 


t .025;n-2 £ 



1000 22938 15565656.6 3499 7456 



2000 


20047 


8504593. 


3000 


17157 


4589076. 


3745 


15003 


3716012 


4000 


14266 


3818107. 


5000 


11376 


6191686. 


6000 


8485 


11709811 



073 


2586 


5512 


808 


1900 


4049 


. 22 


1710 


3643 


809 


1733 


3693 


073 


2207 


4703 


. 60 


3035 


6467 



.462) (4266) 
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TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-ONE (billions) 



FIGURE 1 




32 



FIGURE 2 
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B. PROCUREMENT IN BUDGET- YEAR-PLUS-TWO VS. RDT&E , N 
IN BUDGET YEAR 



1 . Observer Variables , Variab 1 esi n Gene r alized 



Difference Form (r=.446), and Ex Post Forecast 



FY 


X 


X* 


FY 


Y 


y* 


A 

Y t + 1 


62 


3025 




64 


16449 




— 


63 


3394 


2044 . 85 


65 


14580 


7243 . 746 


15552 


64 


3473 


1959.276 


66 


18687 


12184.32 


15010 


65 


3087 


1538.042 


67 


19380 


11045.598 


18278 


66 


3332 


1955.198 


68 


15271 


6627.52 


17165 


67 


3961 


2474.928 


69 


13822 


7011.134 


13560 


68 


3684 


1917 . 294 


70 


14801 


8636.388 


14814 


69 


4075 


2431.936 


71 


13405 


6803.754 


13497 


70 


4011 


2193.55 


72 


14510 


8531.37 


13687 


71 


3718 


1929.094 


73 


13744 


7272 . 54 


15081 


72 


3865 


2206.772 


74 


12307 


6177 . 176 


13793 


73 


3841 


2117.21 


75 


10879 


5390 . 078 


13458 


74 


3755 


2041.914 


76 


12332 


7479 . 966 


13078 


75 


3893 


2218 .27 


77 


14787 


9286.928 


13124 


76 


3991 


2254 .722 


78 


15122 


8526.998 


14095 


77 


4297 


2517.014 


79 


13919 


7174.588 


13350 




2. OLS Regression: Y 


= 28558.594 - 3.7531 


X 




3. Regression Line Adjusted for 


1 Serial Correlation 



Y 

Y 

/\ 

Y 



t+1 
t+1 
t + 1 



rY + a(l-r) + bX x - brX t 

(.446) (13919) + 15188.2496 - 3.4 09 8X^ + 1 + (3.4098) (.446) (4297) 
27930.87573 - 3.4098X t+1 



4. Data Table: 



X t+1 


/\ 

Y t+1 


S f 2 


1000 


24521 


15188245 


2000 


21111 


8056099 


3000 


17701 


4147349 


3713 


15270 


3328866 


4000 


14292 


3461967 


5000 


10882 


5999981 


6000 


7472 


11761382 



. 025 ; n-2 S v 



69 


3488 


7482 


446 


2540 


5449 


016 


1823 


3910 


905 


1633 


3503 


396 


1665 


3571 


585 


2192 


4702 


58 


3070 


6585 
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TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-TWO (billions) 



! 



FIGURE 3 
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RDT&E ,N (billions) TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-TWO (billions) 



FIGURE 4 
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C. PROCUREMENT IN BUDGET- YEAR-PLUS-THREE VS. RDT&E , N 
IN BUDGET YEAR. 



1. Observed Variables, Variables in Generalized 





Difference Form 


(r= . 823 ) , and Ex 


Post Forecast. 


FY 


X 


X* 


FY 


Y 


Y* 


i — 1 
+ 
-P 

< >H 


62 


3025 




65 


14580 












63 


3394 


904.425 


66 


18687 


6687.66 


15264 


64 


3374 


679 . 738 


67 


19380 


4000.599 


16001 


65 


3087 


228.721 


68 


15271 


-678.74 


16580 


66 


3332 


791.399 


69 


13822 


1253.967 


15392 


67 


3961 


1218.764 


70 


14801 


3425 . 494 


15866 


68 


3684 


424 . 097 


71 


13405 


1223 . 777 


13573 


69 


4075 


1043.068 


72 


14510 


3477.685 


14838 


70 


4011 


657.275 


73 


13744 


1802.27 


14243 


71 


3718 


416.947 


74 


12307 


995.688 


12676 


72 


3865 


805.086 


75 


10870 


750 . 339 


13006 


73 


3841 


660.105 


76 


12332 


3378.583 


11266 


74 


3755 


593.857 


77 


14787 


4637.764 


12203 


75 


3893 


802.635 


78 


15122 


2952.299 


15037 


76 


3991 


787.061 


79 


13919 


1473.594 


15253 


2 


. OLS Regression: 


Y = 23284.4885 


- 2.3904 X 




3 


Regression Line 


Adjust 


ed for Serial Correlation: 


i — 1 

+ 

*P 

< >H 


— 2T 


Y + a(l-r) + 


bX ttl 


- brX. 
t 






i — 1 

+ 

< 


= ( 


. 823) (13919) - 


261.28 


79 + 3 . 8988X . - ( 3 . 898 8 ) ( . 823 ) 

t+ 1 


/s 

Y t+1 


= 


1611.922088 + 


3.8988 


X ttl 






4 


Data Table: 










X t + 1 




A 

Y t + 1 


s . 2 

r 


S 

V 


t .025;n-2 S v 



1000 


2287 


25236249 . 15 


2854 


6164 


2000 


6186 


12816433.19 


2034 


4393 


3000 


10084 


6110380.133 


1404 


3032 


3674 


12712 


4813815.148 


1246 


2692 


4000 


13983 


5118089.976 


1285 


2776 


5000 


17882 


9839562.719 


1782 


3849 


6000 


21781 


20274798 . 36 


2558 


5525 
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TOTAL PROCUREMENT IN BUDGET-VEAR-PLUS-THREE (billions) 



FIGURE 5 
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RDT&E , N (billions) TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-THREE (billions) 



FIGURE 6 



22.5 



Y t+ i = .823Y t - 261.2879 + 3.8983X t+1 - (3.8988) ( . 823) X t 
R 2 = .434 

KEY: 



20.0 1 

17.5 , 

15.0 , 

12.5 . 

10.0 « 



OBSERVED DATA * 
FORECASTED DATA 




65 67 69 71 73 75 77 79 

TIME: BUDGET-YEAR-PLUS-THREE, FY... 
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D. PROCUREMENT IN BUDGET- YEAR-PLUS-FOUR VS. RDT&E , N 
IN BUDGET YEAR: 



1. 


Observed Variables, Variables 


in Generalized 




Difference Form (r=.600), and 


Ex Post 


Forecast : 


FY 


X 


X* 


fy 


Y 


Y* 


/N 

Y t+1 


62 


3025 


— 


66 


18687 


— 


— 


63 


3394 


1579 


67 


19380 


8167 . 8 


16676 


64 


3473 


1436.6 


68 


15271 


3643.0 


17058 


65 


3087 


1003.2 


69 


13822 


4659.4 


14490 


66 


3332 


1479.8 


70 


14801 


6507.8 


13734 


67 


3961 


1961. 8 


71 


13405 


4524.4 


14436 


68 


3684 


1307.4 


72 


14510 


6467 


13443 


69 


4075 


1864 . 6 


73 


13744 


5038 


14238 


70 


4011 


1566 


74 


12307 


4060.6 


13708 


71 


3718 


1311.4 


75 


10879 


3494 . 8 


12785 


72 


3865 


1634 . 2 


76 


12332 


5804 . 6 


12005 


73 


3841 


1522 


77 


14787 


7387.8 


12850 


74 


3755 


1450.4 


78 


15122 


6249.3 


14305 


75 


3893 


1640 


79 


13919 


4845 . 8 


14552 


2. 


OLS 


Regression: Y = 


29141.2369 - 4.0109X 


3. 


Regression Line Adjusted for 


Serial Correlation 


A 

Y t+l 


= rY 


t + a(l-r 


) + bx t+1 


- brX t 






/N 

Y t+i 


= (• 


600) (13919) + 5089. 


0391 + . 2376X t+1 - 


( . 2376) ( . i 


Y t+i 


= 12885.45302 


+ .2376 


X t + 1 






4. 


Data Table: 










X t+1 


Y 


t+1 


S f 2 




S v 


t .025;n- 


1000 


13123 


21442577. 


92 


3704 


8072 


2000 


13361 


10722192. 


65 


2620 


5708 


3000 


13598 


4985714. 


989 


1786 


3892 


3651 


13753 


3929622. 


474 


1586 


3456 


4000 


13836 


4233144 . 


939 


1646 


3687 


5000 


14073 


8464482. 


5 


2328 


5072 


6000 


14311 


17679727 . 


67 


3364 


7330 
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TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-FOUR (billions) 



FIGURE 7 



22.5 



20.0 



17.5 



15.0 - 



12.5 - 



10.0 



7.5 ■ 



5.0 « 




DATA POINTS 



r 



\ 

•••\ 



KEY : 

XXX 




OLS LINE — ■ ' 
ADJUSTED REGRESSION LINE 
95 % CONFIDENCE BANDS 

X 



o 1.0 2.0 3.0 4.0 5.0 6.0 

RDT&E,N in BUDGET YEAR (billions) 
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RDT&E.N (billions) TOTAL PROCUREMENT IN BUDGET-YEAR-PLUS-FOUR (billions) 



FIGURE 8 
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E. TOTAL PROCUREMENT IN PROGRAM YEARS VS. RDT&E , N 
IN BUDGET YEAR 

1 • Observed Variables, Variables in Generalized 

Difference Form (r=.363), and Ex Post Forecast . 



FY 


X 


X* 


FY 


62 


3025 


— 


63-66 


63 


3394 


783.425 


64-67 


64 


3473 


543.978 


65-68 


65 


3087 


89.801 


66-69 


66 


3332 


667.919 


67-70 


67 


3961 


1085 . 484 


68-71 


68 


3684 


265.657 


69-72 


69 


4075 


895.708 


70-73 


70 


4011 


494.275 


71-74 


71 


3718 


256.507 


72-75 


72 


3865 


656.366 


73-76 


73 


3841 


505.505 


74-77 


74 


3755 


440.217 


75-78 


75 


3893 


652 .435 


76-79 



Y Y* Y t+1 



69344 






— 


69100 


9256 . 


. 128 


66403 


67921 


8287. 


.7 


66729 


67164 


8548 . 


.177 


66732 


63277 


5314. 


.468 


64780 


57302 


2693. 


. 949 


60489 


56541 


7089. 


. 374 


57173 


56463 


7668 . 


. 117 


55101 


53969 


5341. 


. 431 


55935 


51443 


4867. 


. 753 


54317 


49266 


4870. 


. 691 


51239 


50308 


7791. 


. 442 


49699 


53123 


9707. 


.196 


50745 


56163 


10317 . 


.851 


52698 



2. OLS Regression ; Y = 122325.6933 - 17.4351 X 

3 . Regression Line Adjusted for Serial Correlation 



Y 

A 

Y 

Y 



t+1 

t+1 

t+1 



rY + a(l-r) + “ brX t 

( . 86 3) (56163) + 3317.3507 - 2. 2449X t+1 + (2.2449) (.863) (3893) 
64328.11819 - 2.2449 X t+1 



4. Data Table 



X t+1 


A 

Y t + 1 


s 2 

b f 


1000 


62083 


114633753 


2000 


59838 


57321707 


3000 


57593 


26654035 


3651 


56132 


21008079 


4000 


55349 


22630734 


5000 


53104 


45251807 


6000 


50859 


94517252 





S 

V 


t .025;n- 


4 


5409 


11786 


38 


3825 


8335 




2608 


5683 


12 


2316 


5046 


74 


2403 


5237 


08 


3398 


7405 


04 


4912 


10702 
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TOTAL PROCUREMENT IN PROGRAM YEARS (billions) 



FIGURE 9 



90. 0* 



80. 0* 



70.0. 



60. 0« 



50.0’ 



40.0- 



30.0. 



20 . 0 . 



10 . 0 . 



0 . r 

0 



\ 




\ „ 




KEY : 

DATA POINTS x x * 

OLS LINE 

ADJUSTED REGRESSION LINE 
95 % CONFIDENCE BANDS 





64328.11819 - 2.449X t+1 



\ 



\ 



1.0 



X 

1 _ 

2.0 3.0 4.0 5.0 

RDT&E ,N in BUDGET YEAR (billions) 



■ w - 

6.0 



44 



RDT&E ,N (billions) TOTAL PROCUREMENT IN PROGRAM YEARS (billions) 



FIGURE 10 
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F. PAMN (WPN&APN ) IN PROGRAM YEARS VS RDT&E , N 
IN BUDGET YEAR 

1 . Observed Variables, Variables in Generalized 
Difference Form (r=.821), and Ex Post Forecast 



FY 


X 


X* 


FY 


Y 


Y* 


Y t + 1 


62 


3025 


— 


63-66 


24781 




— 


63 


3394 


910.475 


64-67 


25610 


5264 . 799 


24046 


64 


3373 


686.526 


65-68 


26062 


5036.19 


25004 


65 


3087 


235.667 


66-69 


26323 


4926 . 098 


25932 


66 


3332 


797.573 


67-70 


24090 


2478.817 
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PAMN, APN, WPN IN PROGRAM YEARS (billions) 



FIGURE 11 
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RDT&E.N (in billions) PAMN, APN, WPN (billions) 



FIGURE 12 
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G. TOTAL PROCUREMENT IN THE PROGRAM YEARS VERSUS 
RDT&E , N IN THE BUDGET YEAR, USING ELEVEN MOST 
RECENT DATA POINTS ONLY 

1. Observed Variables, Variables in Generalized 



Difference Form (r-.640) , and Ex Post Forecast 
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TOTAL PROCUREMENT IN PROGRAM YEARS (billions) 



FIGURE 13 
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RDT&E,N (billions) TOTAL PROCUREMENT IN PROGRAM YEARS (billions) 



FIGURE 14 
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H. TOTAL PROCUREMENT IN PROGRAM YEARS VERSUS 
RDT & E , N IN BUDGET YEAR, USING EIGHT MOST 
RECENT DATA POINTS ONLY 



1. Observed Variables, Variables in Generalized 





Difference Form (r=.501), and Ex Post Forecast 
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RDT&E.N (billions) TOTAL PROCUREMENT IN PROGRAM YEARS (billions) 



FIGURE 16 
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V. THE CONCLUSIONS 



A. ANALYSIS OF REGRESSION RESULTS 

Discussion of the regression results will proceed in 
generally the same sequence as the presentations in the 
previous section. Since the explanatory variable of each 
regression is RDT&E,N in the budget year, third-order headings 
which follow will refer only to the dependent variables of 
the regressions to be discussed. 

1. Forecasting A Single Year's Procurement 

Sections IV. A, IV. B, IV. C, and IV. D present the 
results of the analyses of the relationships between current 
RDT&E/N and a single year of total Navy procurement one, 
two, three, and four years in the future, respectively. 

The results are not very impressive. Ex post forecasts 
results in R statistics of .383, .478, .434, and .492, 

indicating that in each case less than half of the procure- 
ment variance is "explained" by the earlier effort in RDT&E. 

In addition, the widths of the 95% confidence intervals 
about these models' point estimates are excessive. The 
average value of the dependent variable during the period 
under study is approximately $15000 million, and these 
models provide confidence intervals of between $5384 million 
and $7286 million in width (both sides). Such statistical 
accuracy would appear to be of little use, even at the 
95% level of significance. Finally, these four regressions 
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appear to have very little relation to one another. 

The adjusted regression lines have a sharply negative 
slope for the first two analyses. The third year's 
adjusted regression line is sharply positive in slope, 
and the fourth year's regression provides a line of only 
slightly positive slope. One would expect either more 
consistency among these regressions or some sort of trend. 
Neither characteristic is in evidence, and the absence of 
any pattern suggests that the time period covered by the 
dependent variable (one year) may be too short. 

2 . Forecasting Total Procurement in the Program Years 

Section IV. E displays the results of the analysis 

of the relationship between RDT&E,N in the budget year 

and total procurement in the following four years (the 

program years) collectively rather than individually. The 

2 

ex post forecast in this case results in a R of .896 and 

2 

a 95% confidence interval of $10092 million. The R 
statistic is impressive in that nearly ninety percent of 
the variation of the dependent variable is "explained" by 
the model, and the confidence interval, when compared to 
the average value of the dependent variable during the 
period studied, $58670 million, suggests that the statis- 
tical reliability of this model is superior to those pro- 
duced by the single year analyses. 

In spite of the apparent success of this analysis, 
one serious difficulty remains to be discussed: the nega- 

tive slope of the adjusted regression. The OPNAV/NAVCOMPT 
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analysis at the very least implied that the relationship 
between RDT&E and future procurement was direct in nature; 
that the more spent now on RDT&E, the more we could expect 
to spend on future procurement. However, both the regression 
results and visual inspection of the data plotted over 
time (Figure 10) reveal that, on the macro level since 
1962, the overall trend in total Navy procurement has 
been negative, while the trend in RDT&E, N has been posi- 
tive. This conflict leads logically to expansion of the 
study in two directions, both of which will be discussed 
in the sections to follow. 

3 . Forecasting Aircraft and Missile (PAMN, APN , 

WPN) Procurement in the Program Years 

In order to explore the possibility of a closer 
and/or more direct relationship existing between RDT&E 
and selected future procurements, the analysis presented 
in Section IV. F was conducted. Of the five current Department- 
of-the-Navy procurement accounts, PAMN (divided into WPN 
and APN since 1974) was thought to be probably the most 
influenced by research and development. Thus, an analysis 
of budget year RDT&E, N in relation to program years’ 
collective PAMN might prove revealing. 

The analysis resulted in ex post forecasts and a 

R statistic of .765 and a confidence interval of $3695 

million in comparison with average value of the dependent 

2 

variable of $22384 million. This R value is a little 
less than that of the previous analysis , but the confidence 
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interval is a little narrower. The indication is that this 
relationship is neither significantly better nor signifi- 
cantly worse than that matching RDT&E with total procure- 
ment. In addition, the model produced has a negative 
slope as did the previous regression. Thus, the expected 
results did not materialize. Although further attempts at 
factoring the data may well yield more useful results, 
this study explores another avenue. 

4 . Total Procurement in the Program Years 

(Reducing the Time Span) 

In attempting to forecast by means of time-series 
data, there always exists a danger that one or more of the 
basic relationships among the variables involved may have 
changed during the period under study. Concern for this 
danger seems particularly appropriate with regard to this 
analysis (specifically, the analysis presented in Section 
IV. E above) . The time period studied includes the Vietnam 
War and the extraordinary impact it must have had on speci- 
fic Navy procurement actions and on the entire Navy budget 
in general. In addition, the very nature of changing tech- 
nology provides a logical basis for arguments that earlier 
data (e.g. data prior to 1970 or 1973) are not valid; and 
visual inspection of the data tends to support this con- 
tention. Note that the last few data-point relationships 
seem to indicate a direct relationship between the variables 
as RDT&E begins to increase, procurement increases also. 

See Figure 12. 
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In response to this, regressions were conducted 
using fewer data points; that is, earlier data were ignored. 
Sections IV . G and IV . H present analyses which use only the 
previous eleven and eight data pairs, respectively. The 
relationship is again between RDT&E,N in the budget year 
and total Navy procurement in the program years . The 
results are interesting in that the slope of the adjusted 
regression lines become more positive as more early data 
are ignored; this indicates that the supposition that the 
basic relationship has changed may have some validity. At 
the same time, however, the width and shape of the 95% 
confidence interval appear to have deteriorated to the point 
of making forecasts much less reliable. If the regression 
resulting from an analysis of only the last eight data 
points (IV. H) is the most representative of the true relation- 
ship between the variables, then there is no alternative 
but to accept a 95% confidence interval which is $13574 
million wide at best (i.e. the next explanatory variable 
is equal to the average of the previous eight) and which 
will increase rapidly as the next explanatory variable 
varies from the average. 

2 

Note also that the R statistics for the ex post 
forecasts conducted using only the most recent eleven and 
eight data points fall to .767 and .508, respectively. 

Although this indicates a deterioration in the goodness 
of fit, the amount of variation in the observed dependent 
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variable is also a good deal less in these regressions than 

in those conducted with all fourteen data points. Since 
2 

the R statistic is constructed using the squares of the 

deviations about the mean of the observed values of the 

dependent variable, it becomes more difficult, proportionately, 
• • 2 

to acnieve the same R statistic when these deviations are 
smaller. In other words, the relatively low R^ statistics 
may not in these cases be cause for rejection of these 
regressions . 

An additional interesting result occurs when com- 
paring the predictions of the three different models for 
the same reasonable value of the explanatory variable, 
for instance $4000 million (by reasonable, we mean in the 
range of experience of the explanatory variable) . The 
point estimates (Y produced by the fourteen-, eleven-, 

and eight-data-point regressions are $55349 million, $54379 
million, and $55064 million, respectively. Thus, the point- 
estimate forecasts of the models are so close as to make 
their differences negligible. This is partially explained 
by the fact that the three models all have the most recent 
eight data points in common and that, by design, the more 
recent data have more impact on the model. Unfortunately, 
neither the proximity of the estimates nor the explanation 
for it helps to resolve the question of the true nature 
of the relationship between these two variables. The point 
estimates may be close, but each model responds quite 
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dif f ercntly to changes in the explanatory variable. Thus, 
the choice of model is still crucial to the budget analyst, 
who is most often faced with decisions of an incremental 
nature . 

B. CLOSING STATEMENTS 

1 . Suggestions for Further Study 

Although necessarily limited, this study may well 
provide a stepping-off point for further analyses. It 
would seem logical that the predictability of future 
procurement could be improved if additional explanatory 
variables were included in the model (e.g. GNP , total DOD 
budget, total Federal Budget, the unemployment rate). 

Lack of sufficient and relevant time-series data might in 
this way be overcome by pooling whatever time-series data 
are available with cross-section data. Of course, the 
methodology of multi-variable forecasting and the process 
of handling error disturbances are considerably more sophis- 
ticated than those used in this thesis. However, in view 
of the results achieved with just RDT&E, more useful and 
accurate predictions might be expected. 

2 . Hypothesis 

With regard to the general working hypothesis 
proposed at the outset, it is concluded that there does 
exist a statistically valid predictive relationship between 
current Navy research and development and future Navy pro- 
curement, taking the program years as a whole. Whether 
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the relationships discovered in this thesis are close 
enough to prove useful to those who shape future Navy 
budgets remains open to question. Probably the most 
common failing of forecasting with time-series data is due 
to an unv/arranted emphasis on the past rather than the 
future. Additionally, the temptation to attribute cause- 
and-effect qualities to purely predictive relationships is 
always present. The quality and usefulness of the models 
developed above or those which may follow should be based 
upon how well they predict the future , and, if a basic 
relationship can change, it must always be kept in mind that 
during the period being predicted, it_ just might 1 
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APPENDIX A 




THE OPNAV MEMORANDUM 



DEPARTMENT OF THE NAVY 

OFFICE OF THE CHIEF OF N AVAL. OPERATIONS 
WASHINGTON, D C. Z0350 

tN «CFt_Y REFER TO 

Ser 922E21/587526 

24 AFP. 1973 



MEMORANDUM FOR SUPERINTENDENT, NAVAL POSTGRADUATE SCHOOL 

Subj : Proposed Study Subject for Financial Management 

Students 

Enel: (1) Discussion Paper 

The purpose of this memorandum is to recommend a subject 
for study by your financial management students. I believe 
the study would provide results useful to those of us involved 
in shaping future Navy budgets and be an excellent project for 
a master ’ s thesis. 



The ever increasing complexity and cost of modern weapons 
systems is a familiar theme and we are indeed seeing more and 
more instances where the development of a promising new system 
is terminated simply because its predicted procurement costs 
make the system non-af fordable . Nonetheless, the research and 
development program for the Department of Defense continues to 
receive increased emphasis during each year's budget delibera- 
tions. For example, the Navy's Research, Development, Test 
and Evaluation (RDT$E) appropriation for FY 1979 is funded at 
a level which provides a program real growth of approximately 
six percent over the FY 1973 level. In order to project the 
magnitude of future Navy budgets that can be anticipated as a 
result of the continued emphasis in research and development, 
my staff recently prepared an analysis of che potential procure- 
ment costs inherent in the on-going R§D programs presently 
funded in the RDT5E appropriation. The objective of the analysis 
was to develop, if possible, a prediction model that would help 
answer the question .... ’’Given a certain level of funding in 
research and development programs, what follow-on procurement 
costs can be expected?” 



Although the analysis performed by my staff was intentionally 
limited, it did produce a rough indicator of approximately four 
dollars of procurement costs for every dollar of RqD, and 
yielded several interesting observations. For example, one 
program had an estimated procurement cost 53 times greater 
than its development cost, while other programs were noted to 
cost more to develop than to procure. Enclosure (1) provides 
additional background and a discussion of the approach used by 
my staff. 
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Subj : Proposed Study Subject for Financial Management 

Students (continued) 

It is anticipated that appropriate research and analysis of 
the historical relationship between known procurement and 
development costs would produce numerical factors that reflect 
the number of procurement dollars required for every dollar of 
R$D. It is envisioned that the development programs would be 
categorized by some appropriate scheme such as functional type 
or mission area, e.g. -- aircraft, missiles, radar systems, sonar 
systems, etc., and that there would be a different factor for 
each category. The factors could be used as models to provide 
at least a rough answer to questions such as.... "If I have $10 
million RDT5E funds invested in advanced surf ace - sonar systems 
development programs, how many follow-on procurement dollars will 
be required?" In order to keep the study effort to a manageable 
level and within the ability of the students to accomplish as a 
thesis project, I recommend that the scope be limited to purely 
financial analysis without regard to the cost-effectiveness of 
the various weapon- systems . 

My point of contact for this proposed area of study is 
Mr. Charles ?. Nemfakos, Associate Director, Budget. I will 
welcome any comments that you or your staff may have concerning 
this project. 



RETURN ON INVESTMENT IN R$D PROGRAMS 



o The best indicator of the amount of future procurement 
dollars associated with new weapons systems now in development 
is an analysis of those programs in the engineering development 
stage of the total research and development program. It is 
these programs that have passed a major milestone in the system 
acquisition decision process and are the most likely to complete 
development and enter production. Many of the programs in 
engineering development already have budget quality estimates 
of their procurement price tag. For those programs whose procure- 
ment cost has not yet been priced out, we can at least obtain 
an order -of -magnitude , ball-park estimate of procurement cost 
using an empirically derived prediction factor. In order to 
derive such a factor for FY 1979, NAVCOMPT analyzed a represen- 
tative sample of programs in engineering development in FY 1979 
for which budget quality estimates of total development and 
total procurement costs were available. From the analysis a 
relationship of approximately four dollars of procurement for 
every dollar of R$D was derived. Using this predictor, we 
filled in the holes in the procurement cost estimates associated 
with our R$D programs in engineering development. 

o The NAVCOMPT analysis used a very small data base of 
approximately 20 programs, as shown in Tab 1. These programs 
were ail in the FY 19"9 budget and in the engineering develop- 
ment stage of their development program. Each of these programs 
had progressed to the point that budget -quality estimates of the 
total R$D costs and total procurement costs were available. The 
programs were selected in such a manner as to provide a cross- 
section sample of the various development technologies within 
the total R$D programs. 



o A more accurate analysis of the relationship between 
procurement and development costs would, of course, require 
a larger data base. The minimum requirement is that both total 
R§D and procurement costs be known -- thus the data base is 
largely historical in nature. Prior-year RDT5E budgets will 
necessarily provide the majority of programs in the sample 
as these budgets will contain more programs for which actual 
total R$D and procurement costs are available. Current year 
and budget year budgets also can be analyzed as budge t -quality 
estimates of total R$D and procurement costs can be considered 
as "known" costs. 
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impact on the overall factor is minimal because that program's 
R5D dollar value represented only one percent of the total 
sample's dollar value. 

o The NAVCOMPT analysis considered only total procurement 
and total RqD costs. The possible affect of unit cost and 
production quantities was not analyzed. A more extensive 
study of procurement and R5D cost relationships should consider 
the impact of a large production quantity or a production run 
over a long period of time. 
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SAMPLE PROGRAMS 



1 

Program 


2 

P/R 


3 

FY 79 AMT 
($ in M’s) 


£ 

Fraction 
of Sample 


5 

Weighted 

Factor 


F-18 


5 . 6 


473.6 


.42 


2.35 


Lamps III 


4.7 


124.5 


.11 


.52 


TACTAS 


3.1 


p 

25.2 


.02 


.16 


HARM 


4.3 


43.4 


.04 


.19 


TOMAHAWK 


1.3 


152.1 


.13 


.23 


STD MSL IMP 


3.5 


49.1 


.04 


.14 


TRIDENT I MSL 


2.2 


191.8 


.17 


.37 


SURTASS 


5.3 


6.6 


.006 


.01 


ASP J 


.'6 


10.2 


.009 


.01 


MCLWG 


3.7 


1.9 


.002 


.01 


ABN ASW Pods 


5.7 


1.0 


.001 


.01 


Adaptive BOM 


4.4 


5.3 


.005 


.02 


SPN-42 RADAR 


4.5 


3.2 


.003 


.01 


Spec W/F Craft 


.6 


6.0 


. 005 


, 01 


ADV RADAR WARNING 


33.2 


.6 


.0005 


.02 


Helo Night Vision 


11.3 


5.5 


.005 


. 06 


Air -Air MSL SYS 


, 4 


28.5 


. 03 


.01 


CIC CONV 


3.0 


7.1 


.006 


.05 






1136.4 


1.000 


4.20 



FY 79 Weighted Multiplier (Procurement to RSD)= 4.2 

Page 1 of 2 
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SAMPLE PROGRAMS 



LEGEND 



COLUMN 



DESCRIPTION 



1 Program short title 

2 Ratio of total procurement 
costs to total R$D costs 

3 Amount of dollars budgeted 
in the FY 79 RDT5EN budget 

4 Fraction representing each 
programs' share of the total 
sample 

5 Weighted procurement - to - 
development factor obtained 

by multiplying columns 2 and 4 



Page 2 of 



TAB 1 
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PROCUREMENT 



RESEARCH, DEVELOPMENT, TESY 
& EVALU ATOM, MAVY 
SPY 1979 
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RESEARCH, DEVELOPMENT, TEST AND EVALUATION, NAVY 

($ MILLIONS) 
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APPENDIX B 



GLOSSARY OF TERMS AND ABBREVIATIONS 
Adjusted regression - An ordinary least squares regression 





which has been corrected for the serial 
correlation of the error terms [Refs. 10 
and 12] . 


APN 


Appropriation title; Aircraft Procurement, 
Navy [Ref. 8] . 


Appropriation 


An annual authorization by Act of Congress 
to incur obligations for specified pur- 
poses and to make payments out of the 
Treasury. Synonyms: budget authority 

and new obligational authority [Ref. 5]. 


Authorization 


Substantive law which must be passed by 
Congress prior to any appropriation which 
specifies the amount and purposes for v/hich 
money may be used. This requirement, which 
originally applied only to procurement of 
aircraft, missiles, and ships, has been 
expanded year-by-year to other categories 
until now it includes procurement of 
tracked combat vehicles and other weapons, 
RDT&E, military construction, torpedoes, 
reserve and active duty personnel strength 
[Ref. 4] . 


Autoregression - 


A condition in which the error terms from 
different observations are correlated; the 
effect of an error disturbance in one 
period carries over into another period. 
Synonym: serial correlation [Refs. 10 

and 12] . 


3udget activity - 


- The major subdivisions of each budget 
appropriation account as submitted to 
and approved by Congress (e.g., the budget 
activities as of fiscal year 1974 for 
the RDT&E,N appropriation are: Technology 

base, advanced technology development, 
strategic programs, tactical programs, 
intelligence and communications, program- 
wide management and support) [Ref. 2] . 
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Budget authority 



Budget plan 



Budget year 
Budgeting 

Constant dollars 
Crosswalking 

Current dollars 

DOD program category 

Econometrics 
Economic forecasting 
Ex post forecast 



An annual authorization by act of 
Congress to incur obligations for 
specific purposes and to make payments 
out of the Treasury. Synonyms: appro- 

priation, new obligational authority 
[Ref. 7]. 

That portion of the programming-and- 
financing section of the Federal budget 
request which shows by budget activity 
the amounts for procurement actions 
programmed and the manner in which the 
total amount is being financed [Refs. 

7 and 8 ] . 

The current fiscal year plus one [Ref. 5] 

The process of translating approved 
resource requirements (manpower and 
material) into time-phased financial 
requirements [Ref. 5]. 

Costs expressed in terms of the price 
levels prevailing in the base year 
(FY 1979 in this case) [Ref. 5]. 

The process of transforming or trans- 
lating the budget from DOD program 
category to appropriation budget 
activity and vice versa [Ref. 1]. 

Costs expressed in terms of the price 
levels prevailing when those costs 
were incurred, obligated, and/or 
expended [Ref. 5] . 

- One of the ten major program cate- 
gories outlined by the Department of 
Defense which are objective-, goal-, 
mission-, and/or output-oriented [Ref. 4] 

The development and use of mathematical 
models representing portions of the 
real world [Ref. 12] . 

- The quantitative estimation about 
the likelihood of future events based 

on past and current information [Ref. 12] 

A quantitative estimation about the like- 
lihood of an event during a past time 
period when the value of both explanatory 
and dependent variables are known with 
certainty [Ref. 12]. 
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Expenditure 


A charge against available funds which 
represent the actual payment of cash. 
Synonym: outlay [Ref. 5] . 



New obligational authority (NOA) - An annual authorization 





by Act of Congress to incur obligations 
for specified purposes and to make 
payments out of the Treasury. Synonym: 
appropriation, budget authority 
[Ref. 5]. 


Obligation 


A legal reservation of a specified amount 
of an appropriation for expenditure 
[Ref. 5]. 


OPN 


An appropriation title: Other Procure- 

ment, Navy [Ref. 8]. 


Ordinary least 


squares (OLS) regression - A procedure for 

determining the statistical relationship 
between two variables by calculating 
the equation for a straight line which 
minimizes the sum of the squared 
deviations (observed-calculated) of 
the dependent variable [Ref. 11] . 


Outlay 


A charge against available funds repre- 
senting the actual payment of cash. 
Synonum: expenditure [Ref. 7]. 


PAMN 


An old appropriation title: Procurement 

of Aircraft and Missiles, Navy. (This 

appropriation was divided into APN and 
WPN in 1974) [Ref. 8] . 


PMC 


An appropriation title: Procurement, 

Marine Corps [Ref. 8]. 


Program 


A plan or scheme of action designed for 
the accomplishment of a definite 
objective which is specific as to the 
time-phasing of the work to be done and 
the means proposed for its accomplishment, 
particularly in quantitative terms, with 
respect to manpower, material, and 
facilities requirements. Provides the 
basis for budgeting [Ref. 5]. 


Program years 


The four fiscal years beyond the budget 
year [Ref . 5 ] . 
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Programming 



R 



2 



statistic 



RDT&E , N 



Reimbursable 



Reimbursements 



The progcess of translating planned mili- 
tary force requirements into time-phased 
manpower and material resource require- 
ments [Ref. 5] . 

A statistic which represents the goodness 
of fit of the line provided by the 
regression procedure versus the observed 
occurrence [Ref. 11]. 

An appropriation title: Research, 

Development, Test and Evaluation, 

Navy [Ref. 8] . 

An amount added to the total direct 
program which reflects the value of 
activity conducted in support of other 
governmental agencies or programs [Ref. 7]. 

When authorized by law, amounts collected 
for materials or services furnished to 
the public or other government agencies 
[Ref. 7]. 



Selected Acquisition Report (SAR) - A report prepared for 

the Secretary of Defense which summar- 
izes current estimates of technical, 
schedule, and cost performance of Navy 
development and procurement programs in 
comparison with original plans and the 
current program [Refs. 3 and 6]. 

Serial correlation - The error terms from different 

observations are correlated; the effect 
of an error disturbance in one period 
carries over into another period. 

Synonym: autoregression [Refs. 10 and 12] . 

Time-series data - Observations of the value of explana- 
tory and dependent variables which are 
collected over time [Ref. 12]. 

Total direct program - The total value of actions pro- 
grammed in direct support of the budget 
activities making up an appropriation. 
Synonum: total obligational authority 

(TOA) [Ref. 8]. 
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Total Obligational Authority (TOA) - The total amount of 

funds available for programming in a 
given year, regardless of the year the 
funds are appropriated, obligated, or 
expended. It includes new obligational 
authority, unprogrammed obligational 
authority from prior years, reimburse- 
ments not used for replacement of inven- 
tory in kind, advance funding for programs 
to be financed in the future, and 
unobligated balances transfered from 
other appropriations. Synonym: total 

direct program [Ref. 5]. 

WPN - An appropriation title: Weapons 

Procurement, Navy [Ref. 8]. 
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